Detecting device, detecting method, generating method, computer program, and storage medium

ABSTRACT

Provided are a detecting device, a detecting method, a generating method, and a computer-readable storage medium that allow the user to readily obtain information on the degree of wear for a worn portion in the human-powered vehicle. A detecting device includes a control unit that detects a worn portion in a human-powered vehicle as a target worn portion from a first image including at least a part of the human-powered vehicle and outputs wear information related to a degree of wear for the target worn portion.

CROSS-REFERENCE TO RELATED APPLICATIONS

The subject application claims priority to Japanese Pat. App. Ser. No.2019-111495, filed Jun. 14, 2019, the entire contents of which arehereby incorporated by reference for all purposes.

TECHNICAL FIELD

The present invention relates to a detecting device, a detecting method,a generating method, and a computer-readable storage medium that detecta part of a human-powered vehicle from an image thereof.

BACKGROUND ART

Human-powered vehicles mounted with components including a frontderailleur, a rear derailleur, a seat post, a suspension, or the likehave been known. A technique of making a diagnosis by connecting adiagnostic device to a component of the human-powered vehicle has beenknown (see Patent Documents 1 and 2).

PRIOR ART DOCUMENT Patent Document

[Patent Document 1] U.S. Pat. No. 7,819,032

[Patent Document 2] U.S. Pat. No. 9,227,697

SUMMARY OF INVENTION Problems to be Solved by Invention

Currently, it is required to provide means for allowing the user to morereadily obtain information related to a worn portion of a human-poweredvehicle.

It is an object of the present invention to provide a detecting device,a detecting method, a generating method, and a computer-readable storagemedium that allow the user to readily obtain information on the degreeof wear for a worn portion in the human-powered vehicle.

Means for Solving Problems

A detecting device according to the first aspect of the presentinvention comprises a control unit including processing circuitry. Thecontrol unit is configured to detect, in a first image including atleast a part of a human-powered vehicle, a worn portion of thehuman-powered vehicle that is classified as a worn portion and outputwear information related to a degree of wear for the detected targetworn portion.

According to the present aspect, the user can readily obtain the degreeof wear for a worn portion in the human-powered vehicle from the imageof the human-powered vehicle.

In a detecting device according to the second aspect of the presentinvention, the worn portion is one of a plurality of worn portions inthe first image, the target worn portion is one of a plurality ofdefined target worn portions, and the control unit detects the pluralityof worn portions in the human-powered vehicle from the first image asthe plurality of target worn portions, and outputs associated wearinformation for each of the plurality of detected target worn portions.

According to the present aspect, the user can readily obtain wearinformation for each of the plurality of worn portions.

In a detecting device according to the third aspect of the presentinvention, the worn portion includes a component of the human-poweredvehicle, and the control unit detects a component of the human-poweredvehicle from the first image as the target worn portion.

According to the present aspect, the user can readily obtain the wearinformation of the component from the image of the human-poweredvehicle.

In a detecting device according to the fourth aspect of the presentinvention, the component includes at least one of a brake shoe, a brakepad, a sprocket assembly, a crank assembly, a chain, a tire, a diskbrake rotor, a rim, and a wire.

According to the present aspect, the user can obtain the degree of wearfrom the image regarding at least one of the brake shoe, brake pad,sprocket assembly, crank assembly, chain, tire, disk brake rotor, rim,and wire of the human-powered vehicle.

In a detecting device according to the fifth aspect of the presentinvention, the control unit outputs wear information for the detectedtarget worn portion by a machine learning model trained to output thewear information in response to a run-time input of an image.

According to the present aspect, the detecting device can output thewear information by the trained machine learning model.

In a detecting device according to the sixth aspect of the presentinvention, a training computing device trains the machine learning modelwith training data obtained by labeling training data images includingat least one of a brake shoe, a brake pad, a disk brake rotor, and a rimwith a degree of abrasion.

According to the present aspect, the detecting device can output theinformation on the degree of abrasion from an image of at least one ofthe brake shoe, brake pad, disk brake rotor, and rim.

In a detecting device according to the seventh aspect of the presentinvention, a training computing device trains the machine learning modelwith training data obtained by labeling training data images includingat least one of a sprocket assembly and a crank assembly with a degreeof abrasion.

According to the present aspect, the detecting device can output theinformation on the degree of abrasion from an image of at least one ofthe sprocket assembly and the crank assembly.

In a detecting device according to the eighth aspect of the presentinvention, a training computing device trains the machine learning modelwith training data obtained by labeling training data images includingat least one of a chain and a wire with a degree of extension.

According to the present aspect, the detecting device can output theinformation on the degree of extension from an image of at least one ofthe chain and the wire.

In a detecting device according to the ninth aspect of the presentinvention, a training computing device trains the machine learning modelwith training data obtained by labeling training data images including atire with a depth of ridges.

According to the present aspect, the detecting device can output theinformation on the degree of wear in association with the depth of theridges from an image including the tire.

In a detecting device according to the tenth aspect of the presentinvention, a training computing device trains the machine learning modelwith training data including a plurality of training data imagesobtained when the worn portion is viewed from different angles.

According to the present aspect, the detecting device can generate thetrained machine learning model that outputs the degree of wear for theworn portion in the human-powered vehicle in accordance with themultiple images related to the human-powered vehicle.

In a detecting device according to the eleventh aspect of the presentinvention, at run-time, the machine learning model outputs the wearinformation in response to run-time input of the first image and userinformation including physical information or attribute information of auser of the human-powered vehicle.

According to the present aspect, the detecting device can generate thetrained machine learning model that outputs the degree of wear for theworn portion in the human-powered vehicle in accordance with the imageof the human-powered vehicle and the physical information of the user ofthe human-powered vehicle.

In a detecting device according to the twelfth aspect of the presentinvention, the control unit outputs a second image in which the targetworn portion is featured.

According to the present aspect, the user can intuitively recognize thedetected worn portion.

In a detecting device according to the thirteenth aspect of the presentinvention, the control unit changes a visual emphasis of the target wornportion depending on the wear information.

According to the present aspect, the user can intuitively recognize thedegree of wear in the detected target worn portion.

In a detecting device according to the fourteenth aspect of the presentinvention, the control unit outputs related information related to thedetected target worn portion.

According to the present aspect, the user can obtain the relatedinformation related to the detected target worn portion from the imageof the human-powered vehicle.

In a detecting device according to the fifteenth aspect of the presentinvention, the related information includes at least one of informationon a type of the target worn portion, an installing method for acomponent related to the target worn portion, a removing method for acomponent related to the target worn portion, and an adjusting methodfor a component related to the target worn portion.

According to the present aspect, the user can obtain the information onat least one of the type, installing method, removing method, andadjusting method related to the target worn portion of the human-poweredvehicle from an image including at least a part of the human-poweredvehicle.

In a detecting device according to the sixteenth aspect of the presentinvention, the information on the installing method includes at leastone of information on a component related to the target worn portion forinstalling a component in the worn portion, and information on a toolfor installing or removing a component related to the target wornportion.

According to the present aspect, the user can obtain the information onat least one of the component and the tool corresponding to the targetworn portion in the human-powered vehicle from the image of thehuman-powered vehicle.

In a detecting device according to the seventeenth aspect of the presentinvention, the related information includes information on a replacementto be replaced with a component related to the target worn portion.

According to the present aspect, the user can obtain the information ona replacement corresponding to the target worn portion in thehuman-powered vehicle from the image of the human-powered vehicle.

In a detecting device according to the eighteenth aspect of the presentinvention, the information on a replacement includes information onanother component required when a component related to the target wornportion is replaced with the replacement.

According to the present aspect, the user can obtain from the image ofthe human-powered vehicle the information on the component required ifreplacement with a replacement corresponding to the target worn portionin the human-powered vehicle is performed.

In a detecting device according to the nineteenth aspect of the presentinvention, the related information includes link information to access aweb site for purchasing an item related to the target worn portiondetected.

According to the present aspect, the user can obtain a necessary itemfor the target worn portion with simple operation.

In a detecting device according to the twentieth aspect of the presentinvention, the control unit outputs the related information as text dataand/or graphical data.

According to the present aspect, the user can obtain the relatedinformation related to the target worn portion as text data and/orgraphical data from the image of the human-powered vehicle.

A detecting device according to the twenty-first aspect of the presentinvention further comprises a display unit that displays informationoutput from the control unit.

According to the present aspect, the user can visually recognize thedegree of wear for the target worn portion by the display unit.

In a detecting device according to the twenty-second aspect of thepresent invention, the display unit receives user input of selectedrelated information related to the target worn portion, and the controlunit outputs detailed information of the selected related information.

According to the present aspect, the user can select information aboutthe worn portion to be displayed in a more detailed manner regarding thedegree of wear for the target worn portion detected from the image ofthe human-powered vehicle and obtain the detailed information.

In a detecting device according to the twenty-third aspect of thepresent invention, the display unit is configured to select the targetworn portion as a selected worn portion on a second image in which thetarget worn portion is featured, and the control unit outputs relatedinformation of the selected worn portion.

According to the present aspect, the user can select information aboutthe worn portion to be displayed in a more detailed manner from thesecond image displayed in a highlighted manner on the image of thehuman-powered vehicle and obtain the detailed information.

A detecting device according to the twenty-fourth aspect of the presentinvention further comprises a storage device that stores informationoutput from the control unit.

According to the present aspect, the detecting device can store theinformation related to the degree of wear for the worn portion in thehuman-powered vehicle.

In a detecting device according to the twenty-fifth aspect of thepresent invention, the control unit stores identification information ofthe target worn portion in the storage device in association withinformation related to the degree of wear.

According to the present aspect, the detecting device can store theinformation related to the target worn portion in the human-poweredvehicle in association with the degree of wear.

In a detecting device according to the twenty-sixth aspect of thepresent invention, the control unit stores identification information ofthe target worn portion in the storage device in association withidentification information of a user of the human-powered vehicle.

According to the present aspect, the detecting device can store theinformation related to the target worn portion in the human-poweredvehicle and the degree of wear in association with the information foridentifying the user.

In a detecting device according to the twenty-seventh aspect of thepresent invention, the control unit outputs identification informationof the target worn portion in association with the information relatedto the degree of wear to an external device.

According to the present aspect, the detecting device can output to theexternal device, for example, a cloud server, the identificationinformation of the target worn portion in the human-powered vehicle inassociation with the information related to the degree of wear.

In a detecting device according to the twenty-eighth aspect of thepresent invention, the control unit outputs identification informationof the target worn portion in association with identificationinformation of a user of the human-powered vehicle to an externaldevice.

According to the present aspect, the detecting device can output to theexternal device, for example, a cloud server, the identificationinformation of the worn portion in the human-powered vehicle and thedegree of wear in association with the information for identifying theuser.

In a detecting device according to the twenty-ninth aspect of thepresent invention, the control unit outputs information for prompting auser to input the first image in accordance with a traveling history ofthe human-powered vehicle.

According to the present aspect, the user can more surely obtain theinformation related to the degree of wear for the worn portion.

In a detecting device according to the thirtieth aspect of the presentinvention, the control unit outputs to an external device the firstimage inputted, in association with a traveling history of thehuman-powered vehicle.

According to the present aspect, the detecting device can output to theexternal device, for example, a cloud server, the image including theworn portion in the human-powered vehicle in association with thetraveling history of the human-powered vehicle.

In a detecting device according to the thirty-first aspect of thepresent invention, the control unit outputs to an external device thefirst image as input, in association with traveling environmentinformation indicating traveling environment of the human-poweredvehicle.

According to the present aspect, the detecting device can output to theexternal device, for example, a cloud server, the image including thetarget worn portion in the human-powered vehicle in association with thetraveling environment under which the human-powered vehicle travels.

The present invention may be achieved as a detecting device having theabove-described characteristic elements described above as well as adetecting method of executing each characteristic processing, a computerincluding a processor to execute such characteristic processing, agenerating method for a trained machine learning model, and acomputer-readable storage medium.

A detection method executable by a processor according to thethirty-second aspect of the present invention comprises: detecting aworn portion in a human-powered vehicle from a first image including atleast a part of the human-powered vehicle as a target worn portion, andoutputting wear information related to a degree of wear for the targetworn portion.

According to the present aspect, the user can readily obtain the degreeof wear for a target worn portion in the human-powered vehicle from theimage of the human-powered vehicle.

A method for generating a machine learning model according to thethirty-third aspect of the present invention comprises: creatingtraining data obtained by labeling a plurality of training data images,each including at least a part of a human-powered vehicle, with a wornportion of the human-powered vehicle and a degree of wear, andgenerating, based on the created training data, a machine learning modelthat detects a worn portion of the human-powered vehicle in response toinput of a run-time input image of at least a portion of thehuman-powered vehicle, as a target worn portion from the image and thatoutputs the target worn portion and a degree of wear.

According to the present aspect, the trained machine learning modeloutputs the worn portion in the human-powered vehicle and the degree ofwear can be generated from the inputted image.

A computer-readable storage medium according to the thirty-fourth aspectof the present invention comprises instructions configured to beexecuted by a processor of a computer, to cause the processor to executeprocessing steps of detecting a worn portion in a human-powered vehicleas a target worn portion from a first image including at least a part ofthe human-powered vehicle, and outputting wear information related to adegree of wear for the target worn portion.

According to the present aspect, the user can readily obtain the degreeof wear for a worn portion in the human-powered vehicle from the imageof the human-powered vehicle.

According to the present aspect, the computer program is read by thecomputer, whereby the computer functions as a device that outputs thedegree of wear for the worn portion in the human-powered vehicle inaccordance with the image of the human-powered vehicle.

Effects of Invention

According to the present disclosure, the detecting device allows theuser to readily obtain information on the degree of wear for a wornportion in the human-powered vehicle from the image of the human-poweredvehicle without using a special diagnostic device.

BRIEF DESCRIPTION OF DRAWINGS

FIG. 1 is a block diagram illustrating the configuration of a detectingdevice in Embodiment 1.

FIG. 2 illustrates the outline of a learning model in Embodiment 1.

FIG. 3 is a flowchart showing one example of a detecting processingprocedure performed by the detecting device in Embodiment 1.

FIG. 4 illustrates an output example of wear information displayed on adisplay unit of the detecting device in Embodiment 1.

FIG. 5 is a block diagram illustrating the configuration of a detectingdevice in Embodiment 2.

FIG. 6 illustrates the outline of a learning model in Embodiment 2.

FIG. 7 is a flowchart showing one example of a processing procedureperformed by the detecting device using the learning model in Embodiment2.

FIG. 8 is a continuation of the flowchart of FIG. 7 showing one exampleof the processing procedure performed by the detecting device using thelearning model in Embodiment 2.

FIG. 9 illustrates an output example of wear information displayed on adisplay unit of the detecting device in Embodiment 2.

FIG. 10 illustrates an output example of related information displayedon the display unit of the detecting device in Embodiment 2.

FIG. 11 illustrates another output example of the related informationdisplayed on the display unit of the detecting device in Embodiment 2.

FIG. 12 illustrates the outline of a learning model in Embodiment 3.

FIG. 13 illustrates another example of the configuration of the learningmodel in Embodiment 3.

FIG. 14 is a flowchart showing one example of a processing procedureperformed by a detecting device in Embodiment 3.

FIG. 15 is a block diagram illustrating the configuration of a systemincluding a detecting device and a server device in Embodiment 4.

FIG. 16 is a flowchart showing one example of a processing procedure inEmbodiment 4.

FIG. 17 is a continuation of the flowchart FIG. 16 showing one exampleof the processing procedure in Embodiment 4.

MODE FOR CARRYING OUT INVENTION

The descriptions of the embodiments below are examples of forms that anoutput device according to the present invention can take, though thereis no intention to limit the forms. The output device according to thepresent invention can take forms different from the embodiments, such asforms of modification of each of the embodiments and a combination of atleast two modifications that do not contradict each other.

In the following description of each of the embodiments, the termsindicating directions, such as front, back, forward, backward, left,right, sideways, upper, lower and so on are used with reference to thedirections shown as the user sits in the saddle of a human-poweredvehicle.

Embodiment 1

FIG. 1 is a block diagram illustrating the configuration of a detectingdevice 1 in Embodiment 1. The detecting device 1 is a smartphone in thefirst example. The detecting device 1 is a tablet terminal in the secondexample. The detecting device 1 may be a wearable information terminalthat takes the shape of glasses in the third example. The detectingdevice 1 includes a control unit 100, a storage unit (i.e., a storagedevice) 102, a display unit 104, a communication unit 108, aninput-output unit 110 and an imaging unit 112.

The control unit 100 includes processing circuitry, e.g. a processorutilizing a central processing unit (CPU) and/or a graphics processingunit (GPU). The control unit 100 executes processing by using a memorysuch as a built-in read only memory (ROM), a random access memory (RAM)and so on. The control unit 100 detects a worn portion in thehuman-powered vehicle as a target worn portion in a first imageincluding at least a part of the human-powered vehicle, and outputs wearinformation related to the degree of wear for the target worn portion.As described below, the control unit 100 detects multiple worn portionsof the human-powered vehicle from the first image as multiple targetworn portions, and outputs wear information for each of the multipletarget worn portions.

The storage unit 102 includes a non-volatile memory, such as a flashmemory, for example. The storage unit 102 stores a computer program 1P.The control unit 100 reads out and executes the computer program 1P. Thecomputer program 1P is provided from the parts maker of thehuman-powered vehicle or is delivered from any distribution server, andis installed in the detecting device 1, which is a general purposecomputer. The computer program 1P causes the computer to executeprocessing of detecting a worn portion in the human-powered vehicle as atarget worn portion from a first image including at least a part of thehuman-powered vehicle, and outputting wear information related to thedegree of wear for the target worn portion. The computer program 1P maybe obtained by the control unit 100 reading out a computer program 5Pstored in a computer-readable storage medium 5 and copying it onto thestorage unit 102. Although a CD ROM is depicted in FIG. 1 , it will beappreciated that the computer-readable storage medium 5 is typically anon-volatile memory, such as FLASH memory.

The storage unit 102 stores a machine learning model 1M. The learningmodel 1M is trained on a training data set during a training phase by atraining computing device 114. Although training computer 114 isdepicted as a separate computing device from detecting device 1, it willbe appreciated that in some configurations these two devices could bethe same computing device. The training computing device 114 is incommunication with the detecting device 1 via a network N. The trainedmachine learning model is downloaded to the detecting device via thenetwork N and stored in the computer-readable storage medium 5. Thelearning model 1M is trained to output the wear information for thetarget worn portion in accordance with input of an image. The controlunit 100 outputs the wear information by the trained machine learningmodel 1M. The learning model 1M may be obtained by the control unit 100reading out a learning model 5M stored in the storage medium 5 andcopying it onto the storage unit 102. Although the storage medium 5 isdepicted as a CD-ROM obtained from the training computer as a datasource, it will be appreciated that it may be any suitable non-volatilestorage device, such as FLASH memory, etc. Further, the learning model5M and computer program 5P may alternatively be stored at and downloadedfrom a server device such as training computer 114 via a computernetwork N, instead of being installed using storage media 5. Atrun-time, which is the time at which a user employs the trained machinelearning model on the detecting device 1 to recognize objects in animage of the human powered vehicle, the control unit 100 is configuredto receive a first image as input and output identification informationthat identifies the object as the target object and a confidence valueof the identification of the object as the target object. This outputmay also be stored in storage unit 102, and exported to other computingdevices via storage media 5 or computer network N.

The worn portion includes a component of the human-powered vehicle. Thecontrol unit 100 detects a component of the human-powered vehicle fromthe first image as a target worn portion. The component includes atleast one of a brake shoe, a brake pad, a sprocket assembly, a crankassembly, a chain, a tire, a disk brake rotor, a rim, and a wire.

The storage unit 102 stores in advance wear information related to atarget worn portion in the human-powered vehicle. The storage unit 102stores information related to the type of the worn portion in thehuman-powered vehicle. The type is, for example, the type of a targetworn portion, that is, the type of a component, and includes at leastone of the brake shoe, brake pad, sprocket assembly, crank assembly,chain, tire, disk brake rotor, rim, and wire.

The storage unit 102 stores information to be outputted by the controlunit 100 besides the information stored in advance. The storage unit 102stores identification information of the user.

The identification information of the user includes a name, a nickname,a user ID, an e-mail address, or the like. The storage unit 102 storesuser information related to the user. The user information includes atleast one of physical information and attribute information of the userof the human-powered vehicle. The physical information includes theheight and weight of the user, for example. The attribute information ofthe user is a gender or age, for example. The attribute information ofthe user may be information on a riding skill. The attribute informationincludes information related to a riding style and a life style favoredby the user, for example.

The display unit 104 is a display device such as a liquid crystal panel,an organic electroluminescent display, or the like.

The display unit 104 displays information to be outputted from thecontrol unit 100. In Embodiment 1, the display unit 104 displays therelated information related to a part of the human-powered vehicletogether with the image of the human-powered vehicle imaged by theimaging unit 112.

The display unit 104 includes an operating unit 106 that is an interfacefor accepting operation performed by the user. In the presentembodiment, the operating unit 106 is a touch panel device included inthe display unit 104. The operating unit 106 may be a physical button, adisplay built-in touch panel device, a speaker, a microphone, or thelike.

The communication unit 108 is a communication module that can connect tocommunicate with a public communication network N. The control unit 100can output information to an external device via the communication unit108.

The input-output unit 110 is an interface to be connected to an externalstorage device or communication equipment. The input-output unit 110 is,for example, a universal serial bus (USB) interface.

The imaging unit 112 includes an image pickup device such as acomplementary MOS (CMOS) image sensor, or the like. The imaging unit 112outputs an image imaged by the image pickup device when activated. Theimaging unit 112 images a still image or a dynamic image in accordancewith an instruction from the control unit 100.

The control unit 100 in Embodiment 1 detects a worn portion of thehuman-powered vehicle by using the learning model 1M. The control unit100 inputs a first image obtained by imaging the human-powered vehicleto the learning model 1M, detects the shown human-powered vehicle aswell as a component as a target worn portion of the human-poweredvehicle, and outputs wear information related to the degree of wear forthe detected worn portion. Detection processing using the learning model1M will be described in detail below.

The control unit 100 outputs wear information by the learning model 1Mthat has already been so trained as to output wear information for thetarget worn portion in accordance with input of an image. FIG. 2illustrates the outline of the learning model 1M. The learning model 1Moutputs the identification information of a component corresponding to aworn portion in the human-powered vehicle together with the degree ofaccuracy and the degree of wear by a supervised deep learning algorithmusing a neural network (hereinafter referred to as NN) as illustrated inFIG. 2 . The learning algorithm for the learning model 1M may be anunsupervised learning algorithm or a recurrent neural network (RNN).

As illustrated in FIG. 2 , the NN of the learning model 1M includesmultiple convolutional layers, multiple pooling layers and multiplefully connected layers that are defined by definition data, classifiesthe object shown in the inputted first image in accordance with thefeatures of the inputted first image, and outputs identificationinformation identifying the classification result and the degree ofaccuracy thereof.

The learning model 1M is trained by training data obtained by thecontrol unit 100 labeling first images each including a worn portion ofthe human-powered vehicle collected via the Internet in advance in thedetecting device 1 with the identification information of a worn portionshown in each of the first images and the degree of wear. The learningmodel 1M may have been generated by a model creating device managed bythe business operator of the human-powered vehicle and have already beentrained.

The learning model 1M may be trained in advance by training data basedon rendering images generated according to an application program fordesign related to a component corresponding to a worn portion in thehuman-powered vehicle by the model creating device managed by thebusiness operator of the human-powered vehicle.

The learning model 1M according to the present embodiment is generatedin a generating method of creating training data obtained by labelingmultiple first images each including at least a part of thehuman-powered vehicle with the identification information of a wornportion in the human-powered vehicle and the degree of wear, and ofgenerating by the created training data a learning model that detects,when an image is inputted, a worn portion in the human-powered vehicleas a target worn portion from the image and that outputs the target wornportion and the degree of wear.

In the first example, the identification information of the componentcorresponding to a worn portion that is to be labeled with the image ofthe training data is information for identifying the type of a componentof the human-powered vehicle. In FIG. 2 , the first image including abrake shoe is labeled with “0,” the first image including a brake pad islabeled with “1,” and the first image including a sprocket assembly islabeled with “2.”

In the second example, the identification information of an object thatis to be labeled with the image of the training data corresponds to amodel number of each component of the human-powered vehicle. Here, thefirst images each including a component that are labeled with respectivemodel numbers are used as training data.

The control unit 100 trains the leaning model 1M such that it outputswear information for the target worn portion in accordance with input ofan image. In the first example, the control unit 100 trains the learningmodel 1M by training data obtained by labeling first images eachincluding at least one of a brake shoe, a brake pad, a disk brake rotorand a rim with the identification information of these components foridentifying them as well as the degree of abrasion.

In the second example, the control unit 100 trains the learning model 1Mby training data obtained by labeling first images each including atleast one of a sprocket assembly and a crank assembly withidentification information for identifying the sprocket assembly and thecrank assembly as well as the degree of abrasion.

In the third example, the control unit 100 trains the learning model 1Mby training data obtained by labeling first images each including atleast one of a chain and a wire with identification information foridentifying the chain and the wire as well as the degree of extension.

In the fourth example, the control unit 100 trains the learning model 1Mby training data obtained by labeling first images each including a tirewith the depth of the ridges of the tire.

In the first to fourth examples, the control unit 100 may train thelearning model 1M by the training data including multiple imagesobtained when a worn portion with the same degree of wear is viewed frommultiple different angles.

The learning model 1M may be trained so as to identify all theabove-described brake shoe, brake pad, sprocket assembly, crankassembly, chain, tire, disk brake rotor, rim and wire, or may beseparately trained for each of the first example, the second example,the third example and the fourth example. Here, the learning model 1Mmay be constituted by a classifier for classifying the first image and amodel for outputting the degree of wear for each classification.

The detection processing using the learning model 1M illustrated in FIG.2 will be described with reference to a flowchart. FIG. 3 is a flowchartshowing one example of a detection processing procedure performed by thedetecting device 1. When the user carrying the detecting device 1, whichis a smartphone, or a maintenance staff for the human-powered vehiclecarrying the detecting device 1 activates the computer program 1P, thecontrol unit 100 executes the following processing.

The control unit 100 accepts a first image including a human-poweredvehicle (step S101). At step S101, the control unit 100 activates theimaging unit 112 to accept image output. Alternatively, having storedfirst images acquired by the imaging unit 112 in advance in the storageunit 102, the control unit 100 may read out a selected one of the firstimages from the storage unit 102 to accept the first image at step S101.

The control unit 100 outputs the accepted first image (step S103). Atstep S103, the control unit 100 causes the display unit 104 to displaythe first image.

The control unit 100 inputs the accepted first image to the trainedlearning model 1M (step S105). When the first image is inputted, thelearning model 1 detects one or more worn portions in the human-poweredvehicle as one or more target worn portions, and outputs theidentification information of the target worn portions, the degree ofaccuracy and the degree of wear. The control unit 100 acquires theidentification information outputted from the learning model 1M forwhich the degree of accuracy is equal to or more than a predeterminedvalue and the degree of wear corresponding thereto (step S107).

The control unit 100 outputs wear information related to the target wornportion and the degree of wear (step S109). At step S109, the controlunit 100 causes the display unit 104 to display text data or graphicaldata, such as a character string or an image, indicating the degree ofwear as wear information. The control unit 100 may be so displayed thata character string indicating the wear information is superimposed onthe first image. The wear information may be outputted to the externaldevice via the input-output unit 110, or may be print output or voiceoutput, not limited to be outputted to the display unit 104.

At step S109, the control unit 100 outputs the wear information for eachworn portion in a selectable manner and outputs the wear information forthe selected worn portion. The output example of the wear informationwill be described in detail with reference to FIG. 4 .

The control unit 100 stores in the storage unit 102 the identificationinformation of the target worn portion in association with theinformation on the degree of wear (step S111) and ends the processing.

FIG. 4 illustrates a display example of the wear information displayedon the display unit 104 of the detecting device 1. FIG. 4 illustratesone example of an application screen 140 that is displayed on thedisplay unit 104. The application screen 140 includes the first image142 that is outputted to the display unit 104 at step S103. On theapplication screen 140, the wear information related to the degree ofwear for the target worn portion is displayed as an object 144 includinga character string so as to be superimposed on the first image 142.

In the example in FIG. 4 , the target worn portion is a brake shoe, anda presumed value indicating a reduced amount of the braking surface isdisplayed as wear information. If the target worn portion is a tire, apresumed value of a reduced amount of the ridges is displayed as wearinformation.

If the target worn portion is a disk brake rotor, a presumed value of areduced amount of the braking surface is displayed as wear information.If the target worn portion is a wire, a presumed value of the extensionratio is displayed as wear information.

As illustrated in FIG. 4 , the control unit 100 may display the reducedamount determined from the degree of wear as wear information or maydisplay a presumed value of the remaining amount. If the target wornportion is at least one of the sprocket assembly and the crank assemblyas well, the control unit 100 may similarly display a presumed value.

The control unit 100 may detect a chain or a wire as a worn portion andoutput a recommended replacement time presumed from the extension ratioas wear information. The control unit 100 may detect a tire as a wornportion to display the depth of the ridges of the tire on a percentagebasis as wear information.

The user can readily obtain the degree of wear for a worn portion in thehuman-powered vehicle from the image of the human-powered vehicle.

Embodiment 2

FIG. 5 is a block diagram illustrating the configuration of thedetecting device 1 in Embodiment 2. Since Embodiment 2 is similar in thehardware configuration of the detecting device 1 to Embodiment 1 exceptthat the related information of a target worn portion is stored in thestorage unit 102, common parts are denoted by similar reference codesand detailed description thereof will not be repeated.

The storage unit 102 of the detecting device 1 in Embodiment 2 storesrelated information related to a target worn portion. The control unit100 outputs the related information related to the target worn portion.The related information includes at least one of information on the typeof a target worn portion, an installing method for a component relatedto a target worn portion, a removing method for a component related to atarget worn portion, and an adjusting method for a component related toa target worn portion.

The information on an installing method includes at least one ofinformation on a component for installing a component related to atarget worn portion in the worn portion and information on a toolrequired for installing or removing the component related to the targetworn portion.

The related information may include information on a replacement for thecomponent of a worn portion as related information related to the targetworn portion. The information on a replacement includes information onanother component that is required when the component of the wornportion is replaced with such a replacement. The related information mayinclude link information for allowing the user to access a web site topurchase an item related to the worn portion as related informationrelated to the target worn portion.

The control unit 100 of the detecting device 1 in Embodiment 2 detects aworn portion shown in the first image together with the position withinthe first image in accordance with the learning model 1M. FIG. 6illustrates the outline of the learning model 1M in Embodiment 2. Thelearning model 1M in Embodiment 2 is trained to output the position,within a first image, of an object related to the human-powered vehicleshown in the first image. Here, the learning model 1M is trained as asingle shot multibox detector (SSD).

As illustrated in FIG. 6 , the learning model 1M here splits theinputted first image into multiple channels, and outputs feature maps ofmultiple scales in a stepwise manner after convolutional processing orpooling processing. The learning model 1M outputs the candidate and thedegree of accuracy of the detection range for each feature map outputevery multiple step, collects the candidates for the detection rangethat are outputted for every multiple step while excluding theduplicated candidate, and outputs the detection frame and the degree ofaccuracy, i.e. score, corresponding thereto.

The training data for training the learning model 1M in Embodiment 2also includes the position, width and height of the box indicating therange of the object within the first image. The learning model 1M thatalso outputs the position of an object may be a model in accordance withR-CNN, YOLO, or the like, though not limited to the model in accordancewith SSD.

The learning model 1M for detecting the worn portion inclusive of theposition illustrated in FIG. 6 as well is trained by training dataobtained by labeling first images each including a part of thehuman-powered vehicle with the identification information and the degreeof wear for the worn portion shown in each of the first images anddesignating the position of the worn portion within the first image. Thetraining data may be created by first images collected from the ownersof or maintenance staffs for the human-powered vehicles.

FIGS. 7 and 8 are flowcharts showing an example of a processingprocedure performed by the detecting device 1 using the learning model1M in Embodiment 2. In the processing procedure of the flowcharts shownin FIGS. 7 and 8 , steps common to the processing procedure shown in theflowchart shown in FIG. 4 in Embodiment 1 are denoted by the same stepnumbers, and the detailed description thereof will not be repeated.

The control unit 100 of the detecting device 1 in Embodiment 2 acceptsinput of the identification information and attribute information of theuser (step S121) and accepts a first image (S101).

The acceptance of the identification information at step S121 may beperformed only at the initial activation of the computer program 1P, ormay be performed at every time the detection processing is performed.The identification information of the user may be a name or a nickname.Alternatively, at step S121, the control unit 100 may accept input bythe user selecting any one of the identification information of themultiple users stored in the storage unit 102.

The learning model 1M in Embodiment 2 outputs the identificationinformation of the detected worn portion, the degree of accuracy, thedetection range of the worn portion and the degree of wear in accordancewith the input of the first image at step S105. The control unit 100acquires the identification information for which the degree of accuracyoutputted from the learning model 1M is equal to or more than apredetermined value, the corresponding detection range, and the degreeof wear (step S123).

The control unit 100 changes the highlight method for a target wornportion depending on the wear information. More specifically, thecontrol unit 100 creates a second image in which the target worn portionis featured in accordance with the detected position, the width and theheight, within the first image, that are included in the information onthe acquired detection range (step S125).

The second image created at step S125 is acquired by superimposing a boxsurrounding a component of the worn portion on the first image, forexample. The highlight method changed depending on the degree of wear isidentified by the color, the thickness and the presence or absence ofblinking of the box. For example, the control unit 100 creates thesecond image in which the box is made red, bold or is made to blink ifthe degree of wear for the worn portion is high to require for thecomponent related to the worn portion to be replaced, while it createsthe second image in which the box is made green, thin or is not made toblink if the degree of wear is low to eliminate the need for replacementof the mechanical part. The second image is acquired by superimposing anoutline on the component of the worn portion displayed on the firstimage, for example. The second image is an image of a speech balloonderiving from the component of the worn portion, for example. The secondimage is acquired by superimposing a translucent image over the range ofthe component of the worn portion displayed on the first image, forexample. The second image may include a character string. The color, thethickness or the motion of the outline, the speech balloon, thetranslucent image or the character string may be changed depending onthe degree of wear.

The control unit 100 that outputs the second image in which the targetworn portion is featured outputs the related information related to thedegree of wear for the worn portion together with the second image (stepS127). At step S127, the control unit 100 causes the display unit 104 todisplay the wear information and the second image.

The processing procedure is continued from step S127 in FIG. 7 to stepS129 in FIG. 8 , as indicated by the circled numeral 1 in FIGS. 7 and 8. The control unit 100 reads out the related information related to thewon portion corresponding to the acquired identification informationfrom the storage unit 102 (step S129).

The control unit 100 accepts input of selecting the target worn portionas a selected worn portion on the second image in which the target wornportion is featured (step S131). The control unit 100 outputs therelated information related to the selected worn portion (step S133).The control unit 100 causes the display unit 104 to display the relatedinformation at step S133. The related information may be outputted tothe external device via the input-output unit 110, or may be printoutput or voice output, not limited to be outputted to the display unit104.

Multiple related information may be outputted at step S133. The controlunit 100 accepts input of selecting the related information related tothe worn portion as selected related information (step S135). Thecontrol unit 100 outputs the detailed information of the selectedrelated information (step S137). At step S137, the control unit 100causes the display unit 104 to display the detail of the relatedinformation.

The control unit 100 stores the identification information of the targetworn portion in association with the information related to the degreeof wear in the storage unit 102 (step S111). The control unit 100 storesthe identification information of the target worn portion in associationwith the identification information of the user of the human-poweredvehicle in the storage unit 102 (step S139), and ends the processing. Atstep S137, the control unit 100 may only store the worn portion selectedat step S131. The worn portion is stored in association with theidentification information of the user, which enables storing accordingto the user's favor such as information on which worn portion isweighted for the user.

FIG. 9 illustrates a display example of the wear information displayedon the display unit 104 of the detecting device 1 in Embodiment 2. FIG.9 illustrates one example of an application screen 140 to be displayedon the display unit 104. The application screen 140 includes a secondimage 148 in which the brake shoe and tires corresponding to the wornportions, and the components including the worn portions are featured.The display unit 104 displayed the target worn portions to beingselectable as selected worn portions on the second image 148 in whichthe target worn portions are featured. Any object 150 corresponding tothe target worn portion included in the second image 148 can be selectedvia a touch panel device included in the display unit 104. In theexample in FIG. 9 , the detected two worn portions and the componentsare highlighted so as to be surrounded by the detection frames, and theselectable objects 150 are represented by speech balloons.

FIG. 10 illustrates a display example of the related informationdisplayed on the display unit 104 of the detecting device 1. If the“brake shoe” is selected from the two worn portions displayed on theexample in FIG. 9 , a menu 146 is displayed within the object 150 forthe user to select at least any one of the “installing method,”“removing method,” “adjusting method,” “purchase web site” and“registration” of the brake shoes as related information as illustratedin FIG. 10 . As illustrated in FIG. 10 , the control unit 100 outputsthe related information as text data and/or graphical data, such as acharacter string or an image.

FIG. 11 illustrates another display example of the related informationdisplayed on the display unit 104 of the detecting device 1. FIG. 11 isa display example displayed when the “removing method” of the “brakeshoe” is selected from the menu 146 displayed in the example in FIG. 10. If the “removing method” is selected, information on the tool requiredfor removal is displayed. As illustrated in FIG. 11 , the control unit100 outputs the detailed information of the related information as textdata and/or graphical data, such as a character string or an image.

The user activates the computer program 1P by using a smartphone or atablet terminal, thereby readily obtain the degree of wear for a wornportion and the related information of the worn portion from the imageof the human-powered vehicle. In Embodiment 2, the detecting device 1detects the position of a worn portion as well to thereby display thesecond image with the worn portion featured, which allows the user toreadily recognize the detected worn portion and the related information.

Embodiment 3

In Embodiment 3, the physical information or the attribute informationof the user are used as input information to be input to the learningmodel 1M for outputting wear information related to the degree of wear.The learning model 1M in Embodiment 3 is trained to output the ratioindicating the remaining amount as the wear information related to thedegree of wear. Since the detecting device 1 in Embodiment 3 is similarin configuration to that in Embodiment 1 except for the learning model1M and the details of the processing, common parts are denoted bysimilar reference codes and detailed description thereof will not berepeated.

FIG. 12 illustrates the outline of the learning model 1M in Embodiment3. The learning model 1M is trained by a deep learning algorithm usingan NN similar to the learning model 1M illustrated in FIG. 2 ofEmbodiment 1. The learning model 1M in Embodiment 3 outputs the wearinformation related to the degree of wear in accordance with the inputfirst image and user information including the physical information orthe attribute information of the user of the human-powered vehicle. Thephysical information is the weight, for example, and is inputted at astage of a layer from which the features of the first image areoutputted as illustrated in FIG. 12 .

FIG. 13 illustrates another example of the configuration of the learningmodel 1M in Embodiment 3. The learning model 1M may be configured toinclude a first model using an NN that outputs a worn portion and thedegree of wear in accordance with input of a first image and a secondmodel using an NN that outputs the degree of wear taking the physicalinformation of the worn portion into account in accordance with theidentification information of the worn portion and the degree of wearthat are output from the first model and the physical information. Ifthe learning model 1M in FIG. 13 is used, the control unit 100 acquiresthe identification information of the worn portion from the first modeland acquires the degree of wear from the second model.

FIG. 14 is a flowchart showing one example of a processing procedureperformed by the detecting device 1 according to Embodiment 3. When theuser carrying the detecting device 1, which is a smartphone, or amaintenance staff for the human-powered vehicle carrying the detectingdevice 1 activates the computer program 1P, the control unit 100executes the following processing.

The control unit 100 accepts the physical information or the attributeinformation of the user of the human-powered vehicle (step S301). Thecontrol unit 100 may accept the identification information of the userand the user information via the operating unit 106 of the display unit104, or may read out the identification information of the user and theuser information that have already been stored in the storage unit 102.

The control unit 100 accepts a first image including a human-poweredvehicle ridden by the user (step S303).

The control unit 100 outputs the accepted first image (step S305). Atstep S305, the control unit 100 causes the display unit 104 to displaythe first image.

The control unit 100 inputs the accepted physical information orattribute information of the user and the first image to the trainedlearning model 1M (step S307).

The learning model 1M in Embodiment 3 detects a worn portion of thehuman-powered vehicle as a target worn portion in accordance with theinput of the physical information or the attribute information of theuser and the first image, and outputs the identification information ofthe target worn portion and the degree of wear taking the physicalinformation or the attribute information into account. The control unit100 acquires the identification information of the worn portion and thedegree of wear from the learning model 1M (step S309). By steps S307 andS309, the control unit 100 detects the worn portion of the human-poweredvehicle as a target worn portion.

The control unit 100 outputs the wear information related to the wornportion corresponding to the acquired identification information and thedegree of wear (step S311). At step S311, the control unit 100 causesthe display unit 104 to display the wear information. At step S311, thecontrol unit 100 may display a character string indicating the wearinformation in such a manner as to be superimposed on the first image.The output of the wear information may be outputted to the externaldevice via the input-output unit 110, or may be print output or voiceoutput, not limited to be outputted to the display unit 104.

The control unit 100 stores the identification information of the targetworn portion in association with the information related to the degreeof wear that is outputted at step S311 in the storage unit 102 (stepS313), and ends the processing.

The detecting device 1 in Embodiment 3 can more accurately output thedegree of wear using the physical information of the user of thehuman-powered vehicle as well as the image of the human-powered vehicle.

Embodiment 4

In Embodiment 4, related information is stored in a server device 2 thatcan be connected to communicate with a detecting device 1. The detectingdevice 1 thus acquires the related information from the server device 2.FIG. 15 is a block diagram illustrating the configuration of a systemincluding the detecting device 1 and the server device 2 in Embodiment4. Since a part of the configuration of the detecting device 1 inEmbodiment 4 is similar to that of Embodiment 1 or Embodiment 2, commonparts of the configuration are denoted by similar reference codes, anddetailed description thereof will not be repeated.

The detecting device 1 in Embodiment 4 includes a control unit 100, astorage unit 102, a display unit 104, a communication unit 108, aninput-output unit 110 and an imaging unit 112.

In the storage unit 102 of the detecting device 1 of Embodiment 4, norelated information is stored. The related information stored in theserver device 2 is used. The storage unit 102 stores a traveling historyof the human-powered vehicle of the user of the detecting device 1,which is a smartphone. The storage unit 102 also stores travelingenvironment information indicating traveling environment of thehuman-powered vehicle. The traveling environment includes information onthe type of a road surface for traveling specified by positioninformation and information on the weather. In the first example, thetraveling history and the traveling environment are information acquiredfrom a cycle computer of the human-powered vehicle through theinput-output unit 110 or another wireless communication module.Alternatively, the traveling history and the traveling environment maybe acquired by the control unit 100 of the detecting device 1 performinga measurement in accordance with another computer program.

The communication unit 108 of the detecting device 1 in Embodiment 4 maybe connected to a public communication network N via an access point APusing a wireless communication device complying with Wi-Fi. Thecommunication unit 108 may be a carrier communication module forachieving communication via a carrier network N2.

A server computer is used for the server device 2. The server device 2includes a control unit 20, a storage unit 22 and a communication unit24. The server device 2 will be described using one server computer,though multiple server computers may be used to share the function orprocessing.

The control unit 20 is a processor using a CPU or a GPU.

The control unit 20 executes processing using a memory such as abuilt-in ROM, RAM, or the like.

The storage unit 22 includes a non-volatile memory, for example, a harddisk, a solid state drive (SSD), or the like. The storage unit 22 storesa server program 2P. The control unit 20 reads out and executes theserver program 2P.

The storage unit 22 includes a related information database DB1, a useridentification information database DB2 and an image database DB3. Therelated information database DB1 includes information related to a wornportion that is detected. The related information database DB1 includesat least one of the information on the type of the detected wornportion, an installing method for a component related to the wornportion, an removing method for a component related to the worn portion,and an adjusting method for a component related to the worn portion. Theinformation on an installing method may include at least one of theinformation on a component for installing the component related to thedetected worn portion in the worn portion and the information on a toolrequired for installing or removing the worn related to the detectedworn portion. The related information database DB1 may includeinformation on a replacement to be replaced with the component relatedto the detected worn portion. The information on a replacement mayinclude information on another component that is required when thecomponent at a worn portion is replaced with such replacement. Therelated information database DB1 includes link information for allowingthe user to access a web site to purchase an item related to the wornportion. The user identification information database DB2 includesinformation on the name, nickname, user ID and e-mail address of theuser of the human-powered vehicle. The image database DB3 includesinformation related to an image including a human-powered vehicle.

The communication unit 24 is a communication module that can beconnected to communicate with a public communication network N. Thecommunication unit 24 is a network card for wired connection. Thecontrol unit 20 transmits and receives information with the detectingdevice 1 by the communication unit 24.

FIG. 16 and FIG. 17 are flowcharts showing one example of a processingprocedure in Embodiment 4. When the user carrying the detecting device1, which is a smartphone, or a maintenance staff for the human-poweredvehicle carrying the detecting device 1 activates a computer program 1P,the control unit 100 executes the following processing.

The control unit 100 of the detecting device 1 outputs information forprompting the user to input the first image to the display unit 104 inaccordance with the traveling history of the human-powered vehicle (stepS401). The output destination at step S401 is not limited to the displayunit 104, and may be a voice input-output unit provided in the detectingdevice 1 used for performing voice output.

The control unit 100 accepts a first image including the human-poweredvehicle (step S403). At step S403, the control unit 100 activates theimaging unit 112 to accept image output. Alternatively, having storedfirst images acquired by the imaging unit 112 in advance in the storageunit 102, the control unit 100 may read out a selected one of the firstimages from the storage unit 102 to thereby accept the first image atstep S403.

The control unit 100 outputs the accepted first image (step S405). Atstep S405, the control unit 100 causes the display unit 104 to displaythe first image.

The control unit 100 inputs the accepted first image to the trainedlearning model 1M (step S407). The learning model 1M in Embodiment 4detects a worn portion as a target worn portion in accordance with theinput of the first image, and outputs the identification information andthe degree of wear corresponding to the target worn portion. The controlunit 100 acquires the identification information and the degree of wearcorresponding to the worn portion that are outputted from the learningmodel 1M (step S409).

The control unit 100 outputs the wear information related to the wornportion corresponding to the acquired identification information and thedegree of wear (step S411). At step S411, the control unit 100 displaysthe wear information for respective worn portions in a selectable manneron the display unit 104. The control unit 100 accepts selection of anyof the wear information for the worn portions (step S413).

The control unit 100 transmits a read-out request for the relatedinformation related to the worn portion that includes the identificationinformation of the selected worn portion from the communication unit 108to the server device 2 (step S415).

The server device 2 receives the read-out request for the relatedinformation by the communication unit 24 (step S501), and thecommunication unit 20 reads out the related information of the wornportion corresponding to the identification information of the wornportion included in the read-out request from the storage unit 22 (stepS503). At step S503, the control unit 20 may read out the linkinformation for allowing the user to access the web site to purchase anitem related to the worn portion stored in the storage unit 22. Thecontrol unit 20 transmits the read related information to the detectingdevice 1 (S505).

The detecting device 1 receives the related information transmitted fromthe server device 2 (step S417), and the control unit 100 outputs therelated information of the worn portion corresponding to theidentification information of the target worn portion (step S419).

The control unit 100 accepts selection of any one of the worn portions(step S421), and outputs the details of the related information of theselected worn portion to the display unit 104 (step S423). For detectingdevice 1, the processing procedure is continued from step S421 in FIG.16 to step S423 in FIG. 17 , as indicated by the circled numeral 2 inFIGS. 16 and 17 .

The control unit 100 accepts registration operation through a menu 146being the related information concerning the selected target wornportion (step S425). The control unit 100 outputs the identificationinformation of the target worn portion to the external device (serverdevice 2) in association with the information related to the degree ofwear (step S427). The control unit 100 outputs the identificationinformation of the target worn portion to the external device (serverdevice 2) in association with the identification information of the userof the human-powered vehicle (step S429).

The control unit 100 outputs the inputted first image to the externaldevice (server device 2) in association with the traveling history ofthe human-powered vehicle (Step S431). At step S431, the control unit100 outputs, regarding the selected target worn portion, the first imageaccepted at step S403 to the external device (server device 2) inassociation with the traveling history of the human-powered vehiclestored in the storage unit 102.

The control unit 100 outputs the first image to the external device(server device 2) in association with the worn environment of thehuman-powered vehicle (Step S433). At step S433, the control unit 100outputs, regarding the selected target worn portion, the first imageaccepted at step S503 to the external device (server device 2) inassociation with the traveling environment of the human-powered vehiclestored in the storage unit 102.

The processing procedures at steps S431 and S433 are not necessarilyperformed, and any one of the processing procedures may be performed.

For server device 2, the processing procedure is continued from stepS505 in FIG. 16 to step S507 in FIG. 17 , as indicated by the circlednumeral 3 in FIGS. 16 and 17 . The server device 2 receives theinformation related to the degree of wear in association with theidentification information of the target worn portion (step S507), andthe control units 20 stores the identification information of the targetworn portion and the information on the degree of wear in the storageunit 22 (step S509). The control unit 20 receives the identificationinformation of the target worn portion in association with theidentification information of the user (step S511), and stores theidentification information of the target worn portion in the useridentification information database DB2 of the storage unit 22 inassociation with the identification information of the user (step S513).

At step S513, the control unit 20 may store the correspondence with therelated information. Here, if outputting the link information for a website to purchase an item related to the worn portion as relatedinformation, the control unit 20 may store the purchase history and thepurchase time at this site in the user identification informationdatabase DB2 of the storage unit 22 in association with theidentification information of the user. The storage unit 20 may storethe replacement time of a replacement at the worn portion in the useridentification information database DB2 of the storage unit 22. Thecontrol unit 20 can read out and output the related information on whichthe purchase history is reflected as related information designed foreach user.

The control unit 20 receives the first image in association with thetraveling history (step S515), and receives the first image inassociation with the traveling environment (step S517). The control unit20 stores the first image associated with the traveling history in theimage database DB3 of the storage unit 22 in association with theidentification information of the user (step S518), and stores the firstimage associated with the traveling environment in the image databaseDB3 of the storage unit 22 in association with the identificationinformation of the user (step S519). The control unit 20 notifies thedetecting device 1 of completion of the registration (step S521), andends the processing.

When receiving the notification of registration (step S435), thedetecting device 1 outputs the completion of registration to the displayunit 104 (step S437), and ends the processing.

At step S518, the first image is stored in the server device 2 inassociation with the traveling history, that is, the information on howfar the human-powered vehicle travels. The information stored at stepS518 may be used for notifying the user how much the component is worn.The information stored at step S518, that is, the first image of theworn or dirty portion may be used as training data for retraining thelearning model 1M aside from the first image of a new component at thesame portion.

At step S519, the first image is stored in the server device 2 inassociation with the traveling environment, that is, the information onwhat environment the human-powered vehicle travels. The informationstored at step S519 may be used for retraining the learning model 1M byusing the first image of the component as training data for eachtraining environment.

Embodiment 4 is configured to allow the server device 2 to store therelated information, and thus the related information is read out fromthe server device 2. The server device 2 may also store the learningmodel 1M. The detecting device 1 may acquire the information on theidentification information that is output from the learning model 1M anduse the information for the detection processing. Here, the serverdevice 2 updates the learning model 1M by the first images collectedfrom multiple detecting devices 1, which shows promise for more accuratedetection.

It is to be understood that the embodiments disclosed here isillustrative in all respects and not restrictive. The scope of thepresent invention is defined by the appended claims, and all changesthat fall within the meanings and the bounds of the claims, orequivalence of such meanings and bounds are intended to be embraced bythe claims.

DESCRIPTION OF REFERENCE CHARACTERS

-   -   1 . . . detecting device    -   100 . . . control unit    -   102 . . . storage unit    -   104 . . . display unit    -   106 . . . operating unit    -   108 . . . communication unit    -   110 . . . input-output unit    -   112 . . . imaging unit    -   114 . . . training computing device    -   1P . . . computer program    -   1M . . . learning model    -   2 . . . server device    -   20 . . . control unit    -   22 . . . storage device    -   24 . . . communication unit    -   5 . . . storage medium    -   5P . . . computer program    -   5M . . . machine learning model

The invention claimed is:
 1. A detecting device comprising: a control unit including processing circuitry configured to execute a machine learning model, wherein the control unit is configured to detect via the machine learning model, in a first image including at least a part of a human-powered vehicle, a worn portion of the human-powered vehicle that is classified as a target worn portion, and the machine learning model is trained to output wear information related to a degree of wear for the detected target worn portion in response to a run-time input of the first image, the control unit outputs related information related to the detected target worn portion, and the related information includes at least one of an installing method for a component related to the target worn portion, a removing method for a component related to the target worn portion, and an adjusting method for a component related to the target worn portion.
 2. The detecting device according to claim 1, wherein the worn portion is one of a plurality of worn portions in the first image, the target worn portion is one of a plurality of defined target worn portions; and the control unit detects the plurality of worn portions in the human-powered vehicle from the first image as the plurality of target worn portions, and outputs associated wear information for each of the plurality of detected target worn portions.
 3. The detecting device according to claim 1, wherein the worn portion includes a component of the human-powered vehicle, and the control unit detects a component of the human-powered vehicle from the first image as the target worn portion.
 4. The detecting device according to claim 3, wherein the component includes at least one of a sprocket assembly, a crank assembly, a chain, a disk brake rotor, a rim, and a wire.
 5. The detecting device according to claim 1, wherein a training computing device trains the machine learning model with training data obtained by labeling training data images including at least one of a brake shoe, a brake pad, a disk brake rotor, and a rim with a degree of abrasion.
 6. The detecting device according to claim 1, wherein a training computing device trains the machine learning model with training data obtained by labeling training data images including at least one of a sprocket assembly and a crank assembly with a degree of abrasion.
 7. The detecting device according to claim 1, wherein a training computing device trains the machine learning model with training data obtained by labeling training data images including at least one of a chain and a wire with a degree of extension.
 8. The detecting device according to claim 1, wherein a training computing device trains the machine learning model with training data obtained by labeling training data images including a tire with a depth of ridges.
 9. The detecting device according to claim 1, wherein a training computing device trains the machine learning model with training data including a plurality of training data images obtained when the worn portion is viewed from different angles.
 10. The detecting device according to claim 1, wherein, at run-time, the machine learning model outputs the wear information in response to run-time input of the first image and user information including physical information or attribute information of a user of the human-powered vehicle.
 11. The detecting device according to claim 1, wherein the control unit outputs a second image in which the target worn portion is featured.
 12. The detecting device according to claim 11, wherein the control unit changes a visual emphasis of the target worn portion depending on the wear information.
 13. The detecting device according to claim 1, wherein the information on the installing method includes at least one of information on a component related to the target worn portion for installing the component in the worn portion, and information on a tool for installing or removing the component related to the target worn portion.
 14. The detecting device according to claim 1, wherein the related information includes information on a replacement to be replaced with a component related to the target worn portion.
 15. The detecting device according to claim 14, wherein the information on a replacement includes information on another component required when a component related to the target worn portion is replaced with the replacement.
 16. The detecting device according to claim 1, wherein the related information includes link information to access a web site for purchasing an item related to the target worn portion detected.
 17. The detecting device according to claim 1, wherein the control unit outputs the related information as text data and/or graphical data.
 18. The detecting device according to claim 1, further comprising a display unit that displays information output from the control unit.
 19. The detecting device according to claim 18, wherein the display unit receives user input of selected related information related to the target worn portion, and the control unit outputs detailed information of the selected related information.
 20. The detecting device according to claim 18, wherein the display unit is configured to select the target worn portion as a selected worn portion on a second image in which the target worn portion is featured, and the control unit outputs related information of the selected worn portion.
 21. The detecting device according to claim 1, further comprising a storage device that stores information output from the control unit.
 22. The detecting device according to claim 21, wherein the control unit stores identification information of the target worn portion in the storage device in association with information related to the degree of wear.
 23. The detecting device according to claim 21, wherein the control unit stores identification information of the target worn portion in the storage device in association with identification information of a user of the human-powered vehicle.
 24. The detecting device according to claim 1, wherein the control unit outputs identification information of the target worn portion in association with the information related to the degree of wear to an external device.
 25. The detecting device according to claim 1, wherein the control unit outputs identification information of the target worn portion in association with identification information of a user of the human-powered vehicle to an external device.
 26. The detecting device according to claim 1, wherein the control unit outputs information for prompting a user to input the first image in accordance with a traveling history of the human-powered vehicle.
 27. The detecting device according to claim 1, wherein the control unit outputs the first image input to an external device, in association with a traveling history of the human-powered vehicle.
 28. The detecting device according to claim 1, wherein the control unit outputs the first image as input to an external device, in association with traveling environment information indicating traveling environment of the human-powered vehicle.
 29. A detection method executable by a processor, the method comprising: detecting, by a control unit via a machine learning model, a worn portion in a human-powered vehicle from a first image including at least a part of the human-powered vehicle as a target worn portion; in response to a run-time input of the first image, outputting, by the machine learning model, wear information related to a degree of wear for the target wear portion; and outputting, by the control unit, related information related to the detected target worn portion, wherein the related information includes at least one of an installing method for a component related to the target worn portion, a removing method for a component related to the target worn portion, and an adjusting method for a component related to the target worn portion.
 30. A method for generating a machine learning model, the method comprising: creating training data obtained by labeling a plurality of training data images, each including at least a part of a human-powered vehicle, with a wear portion of the human-powered vehicle and a degree of wear; and generating, based on the created training data, a machine learning model that detects, in response to input of a run-time input image of at least a portion of the human-powered vehicle, a worn portion of the human-powered vehicle in the run-time image as a target worn portion, and outputs wear information related to a degree of wear for the target worn portion, wherein related information related to the target worn portion detected by the machine learning model is output by a control unit, the related information including at least one of an installing method for a component related to the target worn portion, a removing method for a component related to the target worn portion, and an adjusting method for a component related to the target worn portion.
 31. A non-transitory computer-readable storage medium comprising instructions configured to be executed by a processor of a computer, to cause the processor to execute processing steps of: detecting, by a control unit via a machine learning model, a worn portion in a human-powered vehicle as a target worn portion from a first image including at least a part of the human-powered vehicle; in response to a run-time input of the first image, outputting, by a trained machine learning model, wear information related to a degree of wear for the target worn portion; and outputting, by the control unit, related information related to the detected target worn portion, wherein the related information includes at least one of an installing method for a component related to the target worn portion, a removing method for a component related to the target worn portion, and an adjusting method for a component related to the target worn portion. 